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INTRODUCTION 

The purpose of the Learning Abilities Measurement 
Program (LAMP) is to conduct basic research on 
the nature of human learning and performance. 

The ultimate goal of this research is to build 
an improved model-based selection and classifica- 
tion system for the United States Air Force. 
During the first few years of the program, and 
continuing through to the present, researchers 
are developing innovative approaches to ability 
testing (Kyllonen & Christal, in press). In 
conjunction with this framework, new kinds of 
computerized ability tests have been developed 
(Fairbank, Tirre & Anderson, 1987; Tirre & 
Rancourt, 1986; Woltz, 1986; Woltz, 1987). 

LAMP examines individual differences in learning 
abilities, seeking answers to the following 
questions: 

1. Why do some people learn more and better 
than others? 

2. Are there basic cognitive processes 
applicable across tasks and domains that 
are predictive of successful performance, 
or are the behaviors in question more 
involved (e.g., complex problem solving 
behaviors) ? 

3. Which of these processes or learning 
abilities are domain specific and which 
generalize across subject areas? 
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We have used some simple learning tasks to 
determine the elementary cognitive processes 
involved in learning abilities such as: Infor- 
mation processing speed, prior knowledge, and 
working memory capacity (size and activation 
level) . To test the extent of differential 
learning abilities based on these rudimentary 
processes, we need to examine learning in pro- 
gress in complex environments, like intelligent 
tutoring systems (ITS*s), which reflect *real 
world* performance rather than artificial labor- 
atory tasks (like paired associate or rule learn- 
ing) which often do not generalize to the real 
world. There are basically two categories of 
related activities in this research program. 
First, we are concerned with individual differ- 
ences in learners* knowledge and skills. In this 
regard, our aim is to identify more efficient 
and precise methods of individual assessment. 
Second, we are interested in validating models 
of ability organization by (a) estimating indi- 
vidual skill and knowledge levels, (b) estimating 
individual proficiency levels on various learn- 
ing tasks, and (c) relating the two sets of 
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estimates using exploratory and confirmatory 
mathematical modeling techniques such as regres- 
sion analysis and factor analysis. 

We have contracted to have three complex, long- 
term learning tasks (i.e., ITS’s) developed. The 
three tutors teach electronics trouble-shooting, 
flight engineering, and Pascal programming. 

These ITS’s, each requiring about seven days for 
completion of the curricula, are, for the most 
part, based on instruction and test modules from 
operational Air Force training courses. We are 
using another ITS for basic research that has a 
more discovery-oriented learning approach invol- 
ving principles of microeconomics. In addition 
to encompassing economic concepts, "Smithtown" 
(Shute & Glaser, in press) assists the learner in 
becoming more methodical and ’scientific 1 in 
their pursuit of knowledge obtainable from the 
system. Learning parameters estimated from per- 
formance in all of the ITS courses serve as 
intermediate criteria against which measures of 
knowledge and skill acquisition will be evaluated. 
The success of LAMP will ultimately not depend 
on whether we can predict who is more adept at 
acquiring simple facts and rules from the short- 
term tasks, but on whether we can predict who 
will acquire more permanent and complex sets of 
skills characteristic of effective operational 
job performance. Thus, our main concern is with 
validating models of cognitive skills against 
performance in complex learning environments. 

INTELLIGENT TUTORING SYSTEMS AS RESEARCH TOOLS 
We are using intelligent tutoring systems as 


experimental vehicles to determine the set of 
predictor variables effective in predicting 
understanding and learning in complex environ- 
ments. In any intelligent tutoring system, the 
learner interacts with a computer program to 
acquire new information and exercise newly ac- 
quired skills. The program presents problems to 
the student in an adaptive fashion by taking into 
account both the structure of the concepts from 
a subject domain (i.e., the curriculum) and the 
individual learner’s current knowledge and under- 
standing of that subject domain (i.e., the stu- 
dent model) . Such programs can provide a rich 
trace of the individual's learning performance, 
states of knowledge, and rate of progress 
through the curriculum. 

With each ITS, analyzed separately, we begin our 
research by delineating a large set of knowledge 
and performance indicators for a given tutor, 
and then relate these behaviors back to the in- 
dividual cognitive processes as well as to objec- 
tive measures of learning (see Shute, Glaser & 
Raghavan, 1987) . To illustrate, the Pascal pro- 
gramming tutor has general purpose data analysis 
tools which let us specify exactly which per- 
formance or knowledge indicators we want output 
from the extensive student history list. Any 
action or sequence of actions can be specified 
as an ’event’. For example, we can set up any 
event where A, B, C, and D are particular 
actions : 

El > (The student does A and B then (C or D)), or 
E2 : (The student does A or B and (not C)). 

The system computes how many times this sequence 
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occurred, the errors in performance on this 
event, the number of intervening events between 
subsequent occurrences of this event, and so on. 
We can specify very simple actions as events 
(e.g.. The student does A) to more complex series 
of actions to see how the student progresses 
over time. 

Thus, the ITS research can serve as an ideal 
source of intermediate learning criteria against 
which conventional and experimental aptitude 
tests can be validated. For instance, we can 
determine whether processing speed or working 
memory capacity is more important in ascertain- 
ining who will be successful in learning Pascal 
programming, or perhaps it is determined more 
from higher level "planning" types of behaviors 
(Anderson, 1987). 

Intelligent tutoring systems provide us with 
controlled, rich environments to investigate 
individual differences in the acquisition of 
knowledge and skills. In addition, they provide 
us with comprehensive traces of all student 
actions involved in the learning of a given 
subject matter. The tutors consist of complex, 
real world type environments, allowing us to 
abstract so much more information about learning 
than is possible from static paper and pencil 
tests. 

One important consideration in using ITS's is 
that some computer learning environments are 
clearly not suitable for all types of subject 


populations (e.g., discovery worlds). To illus- 
trate, two groups of subjects have been run on 
Smithtown, the intelligent discovery world envi- 
ronment mentioned earlier that embraces the laws 
of supply and demand in a hypothetical marketplace 
(Shute, et al., 1987). Variables such as the 
population or weather can be manipulated, the 
results noted, and principles and laws induced 
from the findings. University students were, for 
the most part, very positive' about it, and said 
things like, "What a fun game... I learned a lot 
about economics". On the other hand, basic Air 
Force recruits (N* 530) were mostly bewildered 
by the environments, typically complaining that, 
"I’ve been lost the whole time!" and constantly 
asking, "What should I be doing?" This is not 
surprising given the different structures and em- 
phases of the two settings (i.e., academic vs. 
military contexts). Given this finding, it 
would be a relatively easy adjustment to make 
the environment more structured for those in- 
dividuals requiring more of a framework for 
learning. 

FUTURE DIRECTIONS 

The tutors will allow us to predict various 
properties of the acquisition process for dif- 
ferent Air Force related knowledge and skills 
from measures developed within the LAMP project. 

In addition, the measurements of the course of 
acquisition and its variability across indivi- 
duals can be used to shape and confirm extensions 
to current theories of knowledge and skill 
acquisition as well as to document the critical 
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individual differences that arise during this 
process. 

Three types of learning progress indices will be 
used to measure different aspects of the course 
of learning. These include measurements of an 
individuals rate, quality and durability of 
learning. Specifically, the three measures are: 
performance criteria (e.g., the number of times 
tutor advice was required), categories of acqui- 
sition trajectories (e.g., change in performance 
speed as a function of practice) and process 
measures (e.g., plans that a subject develops 
for solving a problem) . 

Currently, we are contracting to have intelli- 
gent tutoring systems developed on PC AT-compa- 
tible machines (mini-tutors) . These systems 
will consist of job skills extracted from the 
larger tutors such as: declarative knowledge 

acquisition of electrical circuits, procedural 
knowledge of graph interpretation, and so on. 
These mini-tutors, lasting only 1-3.5 hours 
instead of 7 days, will allow us to refine 
hypotheses and measures with the mini- tutors 
criteria before actually testing them out on 
a large scale. We will be able to more pre- 
cisely analyze the learning of specific pro- 
ductions underlying complex skills. These 
systems will also be considerably more cost 
effective than the larger tutors in terms of 
subject hours and hardware costs. 

SUMMARY 

Assessing individual differences in cognitive 


processes using experimental learning tasks is 
just one aspect of the LAMP effort. Another, and 
more exciting feature, is the mechanism we are 
concurrently using to extend our findings from 
the simpler, often contrived environments to 
more complex, real world types of environments 
via intelligent tutoring systems. Thus, the 
LAMP program and its use of ITS's as experimental 
testbeds represents an innovative twist on an 
old stream of research: investigating individual 

differences in learning as it relates to success- 
ful on-the-job learning and performance. Our 
ITS’s, as intermediate criteria, will enable us 
to assess the same kind of learning as occurs in 
real world tasks, but in controlled environments 
with rich traces of the active, ongoing learning 
processes. This can be contrasted to the paper 
and pencil tests historically (as well as cur- 
rently) used by the Air Force to assess learning 
and abilities. These tests only provide post 
hoc, static measures or depictions of learning, 
with many unanswered questions regarding the 
route to that end. The ITS ? s let us look at a 
range of individual differences in learning from 
simple cognitive processes such as information 
processing speed (and its various components, 
such as encoding, comparing, choosing, retrie- 
ving, attention shifting and memory searching) 
to more complex problem solving processes such 
as means ends analysis and hypothesis generation 
and testing. 
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